Method and apparatus to select the best video frame to transmit to a remote station for CCTV based residential security monitoring

ABSTRACT

A security monitoring system including: an alarm system having detectors for detection of an alarm in a structure; at least one camera for capturing image data inside and/or outside the structure; a processor for selecting a subset of the image data upon the occurrence of the alarm based on a predetermined criteria; and a modem for transmitting the subset of image data to a remote location. The processor ranks each video frame from the image data according to how well each video frame meets the predetermined criteria and the modem transmits a predetermined number of video frames having the best rank to the remote location.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates generally to security monitoringsystems and, more particularly, to a security monitoring system in whicha video frame from video surveillance cameras is selected fortransmission to a remote monitoring station based on a set ofpredetermined criteria.

[0003] 2. Prior Art

[0004] Security monitoring systems of the prior art, particularlyresidential security systems, typically utilize a box that monitorscontact sensors for doors and windows and one or more infra-red sensorsfor area monitoring. When a contact is triggered or an infra-red sensortriggers, an alarm is sounded and a signal is sent via a data link suchas a phone line to a central monitoring site. The central monitoringsite typically initiates a set of phone calls, to the homeowner, to armthe security system and remain inside the residence, such as during thenight and may thereafter leave to get something outside the residence,e.g., the morning paper, thereby triggering a false alarm.

[0005] In view of the prior art, there is a need for a securitymonitoring system, which resolves these and other types of entry/exitconflicts.

SUMMARY OF THE INVENTION

[0006] Therefore it is an object of the present invention to provide asecurity monitoring system which reduces the number of false alarmsinherent in the prior art security monitoring systems.

[0007] It is a further objective of the present invention to provide asecurity monitoring system which transmits image data to a remotemonitoring station for inspection by the monitoring staff to determineif an alarm is a true or false alarm.

[0008] It is yet another objective of the present invention to provide asecurity monitoring system which selects a portion of the image data tothe remote monitoring station based upon a set of predetermined criteriasuch that the bandwidth for transmitting the image data is not exceeded.

[0009] Accordingly, a security monitoring system is provided. Thesecurity monitoring system comprises: an alarm system having means fordetection of an alarm in a structure; at least one camera for capturingimage data inside and/or outside the structure; a processor forselecting a subset of the image data upon the occurrence of the alarmbased on a set of predetermined criteria; and transmission means fortransmitting the subset of image data to a remote location.

[0010] Preferably, the at least one camera is a video camera, the imagedata is video image data, and the subset of the image data is at leastone video frame of the video image data. The transmitted image datashould be such as to allow a monitoring operator to quickly decide ifthe alarm is a true alarm or a false alarm. More preferably, theprocessor ranks each video frame from the image data according to howwell each video frame meets a set of predetermined criteria whichevaluate how useful they are to a monitoring operator in distinguishingfalse/true alarms, and the transmission means transmits a predeterminednumber of video frames having the best rank to the remote location.

[0011] One approach to transmitting image data to the remote location isto simply transmit the frame at the time of the alarm, or a set offrames corresponding to some time interval around the alarm time (e.g.,transmit an image taken 0.5 seconds before the alarm, at the alarm and0.5 seconds after the alarm). However, such a scheme is not guaranteedto catch the cause of the alarm, and even if the cause is caught, it maynot be portrayed in a form quickly interpretable by the monitoringoperator (e.g., the image may catch the heel of an intruder departingthrough a doorway).

[0012] In preferred implementations of the security monitoring system ofthe present invention, the processor subtracts an established backgroundfrom each video frame resulting in a difference region. Thepredetermined criteria that evaluate how useful the image is to amonitoring operator are selected from a group consisting of: howcentered the difference region is in the video frame; how large thedifference region is in the video frame; whether the difference regionconsists of a large difference region or a group of smaller differenceregions in the video frame; the contrast of the difference region in thevideo frame; the lighting condition on the difference region in thevideo frame; and whether a face is detected in the difference region inthe video frame. If the predetermined criteria includes whether a faceis detected in the difference region of the video frame, furtherpredetermined criteria are preferably selected from a group consistingof: how much of the face is visible in the video frame; whether the faceis turned towards of away from the video frame; and whether key featuresof the face are visible in the video frame.

[0013] In alternative implementations of the security monitoring systemof the present invention, the predetermined criteria are selected from agroup consisting of: whether the video frame is blurred; how much skincolor is contained in the video frame; if a person is recognized in thevideo frame; and the lighting condition on a region of motion in thevideo frame.

[0014] In yet another preferred implementation of the securitymonitoring of the present invention, the system further comprises: animage recording system for recording the image data to be analyzed bythe computer vision system wherein the image data is recorded for apredetermined time period before and after the occurrence of the alarm;an analog to digital converter for converting analog image data todigital image data prior to being analyzed by the processor; and acompression means for compressing the subset of the image data prior totransmission to the remote location.

[0015] Also provided are methods for security monitoring of a structurehaving the security monitoring system of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] These and other features, aspects, and advantages of theapparatus and methods of the present invention will become betterunderstood with regard to the following description, appended claims,and accompanying drawings where:

[0017]FIG. 1 illustrates a schematical view of a structure having thesecurity monitoring system of the present invention.

[0018]FIG. 2 illustrates a flow chart outlining a preferred method forutilizing the system of FIG. 1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0019] Although this invention is applicable to numerous and varioustypes of security monitoring systems and image data, it has been foundparticularly useful in the environment of residential securitymonitoring systems and video image data. Therefore, without limiting theapplicability of the invention to residential security monitoringsystems and video image data, the invention will be described in suchenvironment.

[0020] Referring now to FIG. 1, there is illustrated a preferredimplementation of a security monitoring system of the present invention,referred to generally by reference numeral 100. The security monitoringsystem comprises an alarm system 102 having means for detecting anunauthorized individual in a structure 104. Such means can be anyconventional detectors known in the art, such as magnetic door contacts106 for detecting the opening of an entrance door 108 or an infraredmotion detector 110 appropriately positioned to detect the presence ofan unauthorized intruder. Those skilled in the art will recognize thatany such type of conventional detectors may be used without departingfrom the scope or spirit of the present invention.

[0021] If the alarm system 102 is triggered an alarm is sounded and/oran alarm signal is transmitted to a remote monitoring station 112 or alocal police station (hereinafter both are referred to as a “remotemonitoring station”). As discussed above, many of the alarms turn out tobe false alarms in that the triggering of the alarm was unintentional orby an individual who is authorized to be in the structure 104 (e.g., afamily member or maid of a residence). The problem with false alarms canbe resolved by augmenting the alarm system 102 with at least one camera114, and preferably, with a set of inexpensive cameras, such as CCTVvideo cameras 114, 116 for capturing image data of individuals insideand/or outside the structure 104. For those cameras 114, 116 placed in aroom 118 of the structure 104, the cameras 114, 116 are preferablyplaced high in the room 118 and positioned to have as complete a view aspossible. For entrance ways 120 it is advisable not to have the camera114 pointed right at the door 108 unless the camera 114 is equipped witha backlight filter (not shown).

[0022] The alarm system 102 is further augmented with means for storingthe image data captured by the cameras, such as an image recordingsystem 122 and means to transmit the image data 102 a to a remotemonitoring station, which is preferably a built-in function of the alarmsystem 102. It is preferred that the image recording system 122 ispreferably a computer or other processor having a storage device such asa hard drive and an image capture card. However, those skilled in theart will recognize that the image recording system 122 can be of anytype known in the art without departing from the scope and spirit of thepresent invention.

[0023] Furthermore, a compression means 124 can also be provided tocompress the video image data prior to transmission to the remotemonitoring station 112. Such compression and transmission means are wellknown in the art and a detailed description thereof is omitted for thesake of brevity. Compression means 124 can include encoders whichcompress data according to a standard compression algorithm, such asJPEG (for image data) and MPEG (for video data). Transmission means 102a can include any device such as a modem which transmits data via a datalink 126 such as a telephone, ISDN, or coaxial cable line. Upon receiptof the image data, the staff of the remote monitoring station 112 canliterally “see” what caused the alarm and can even perform a comparisonof an individual who caused the alarm with the individuals authorized tobe in the structure 104, as is described in co-pending U.S. patentapplication No. (Docket No. 701609, 13933) which is incorporated hereinby its reference.

[0024] However, given the limitations on transmission bandwidth,particularly from a residence which typically uses a telephone (POTS)connection, it is practical to only send a few video frames from thevideo image data to the remote monitoring station 112. The system 100 ofthe present invention selects a portion of the image data captured bythe camera(s) 114, 116 that is more representative of the cause of thealarm being triggered, thereby filtering out image data that is not ormay not be conclusive as to the determination at hand, namely, if thealarm is a true or false alarm. Since, the remote monitoring station 112uses the transmitted image data to make its decision, the selection ofwhat images to transmit is crucial. The transmitted image data should besuch as to allow a monitoring operator to quickly decide if the alarm isa true alarm or a false alarm. In the case of video image data, theportion thereof to transmit to the remote monitoring site is one or morevideo frames from the video image data.

[0025] To make the selection of what video frame(s) to transmit, thesystem 100 of the present invention first digitizes the video image data(if not already digitized) in an analog to digital (A/D) converter 128and feeds the digitized video image data to a processor 130, which canbe or include a computer vision system 132. The processor 130 analyzesthe digitized video image data and decides what portion (i.e., videoframes) to transmit based on a set of predetermined criteria. Of course,the set of predetermined criteria can comprise of a single criteria. Ifthe alarm system 102 has several monitoring zones, with one or morecameras per zone, the criterion to be used can simply be to compress andtransmit the image data from the zone that triggered the alarm.Similarly, if a video motion detection scheme is used, then only thevideo image data from cameras that trigger a motion detection alarm iscompressed and transmitted to the remote monitoring station 112.

[0026] A more sophisticated set of criteria can include using anestablished background subtraction technique to eliminate parts of thevideo frames from previous video frames that represent a backgroundscene leaving a set of regions which has changed from the backgroundscene. There are many approaches known in the art to do this, such asthat disclosed in A. Elgammal et al., Non-Parametric Model forBackground Subtraction, presented at Int. Conf. Computer Vision &Pattern Recognition, Workshop on Motion, Fort Collins Colo., June 1999.

[0027] Background subtraction provides as output a set of regions of thevideo frame that contain potential intruder images. Some simpleapproaches can be taken to rank whether a specific video frame from avideo camera contains an intruder image of sufficient quality/content tosend to the central monitoring station 112. For example, the morecentered (in the video frame) the regions detected by backgroundsubtraction, the better framed the image shows the intruder, and thebetter the video frame rates for sending to a monitoring station. Othercriteria for rating a video frame in addition to centeredness include,whether one large region versus a set of smaller regions is reported,whether the region shows good contrast, or whether the region is largeversus small.

[0028] Similarly, face detection can be applied to the region with theuse of the computer vision system 132. Such computer vision systems andalgorithms are well known in the art, such as that disclosed in H.Rowley et al., Human Face Detection in Visual Scenes, Advances in NeuralInformation Processing Systems 8, 1996, pp. 875-881 and H. Rowley etal., Rotation Invariant Neural Network-Based Face Detection, Proceedingsof IEEE Conference on Computer Vision and Pattern Recognition, June,1998. Briefly, such computer vision systems 132 look for skin coloramong the pixels of the image data (since skin color has a distinctivehue). If a grouping of skin color pixels is above a threshold (i.e., 20%of the image data), the computer vision system concludes that thegrouping may be a face. If other criteria are met for the grouping, suchas having an elliptical shape and regions which appear to be facialfeatures (e.g., two eyes, a nose, and a mouth), the computer visionsystem concludes that the grouping of pixels is the face of theunauthorized individual. The more visible the face the better the imageranks. More visible can mean: how much of the face is seen (the more thebetter), whether the face is turned towards or away, and whether keyfeatures such as eyes are visible.

[0029] The computer vision system 132 can be used for other imageprocessing techniques to decide which video image data is to betransmitted to the remote monitoring station 112. Such techniques canuse the following criteria to select the portion of the video image datato be transmitted to the remote monitoring station, the video framewhich is least blurred, the video frame that contains the most skincolor information, the video frame that has the best lighting conditionon the region of motion or difference, the video frame in which thepresence of a person is recognized, etc.

[0030] Preferably, a specific time limit is set up around the time ofthe alarm, for example, a predetermined time before (t1 seconds) andafter (t2 seconds) the alarm is triggered. All the video image data fromthe camera(s) 114, 116 in this time period (t1 . . . t2) are analyzedand given a rank based on a set of predetermined criteria which mayinclude any one or more of the above discussed criteria. The top scoringn video frames are sent to the central monitoring station, where n canbe any number. Preferably n=1, i.e., a single video frame is compressedand transmitted to the remote monitoring station 112.

[0031] The methods of the present invention which utilize the preferredimplementation of the security monitoring system illustrated in FIG. 1will now be briefly described with reference to FIG. 2. The method ofFIG. 2 being generally referred to by reference numeral 200. At step202, an alarm is triggered in a structure 104, such as a residence. Thealarm is detected by any conventional means known in the art, such as bydetectors 106, 110 operatively connected to alarm system 102. At step204, the image data, preferably video image data, captured by cameras114, 116 is recorded by the image recording system 122 for a preferredperiod of t1 to t2, t1 being a period in seconds just prior to thetriggering of the alarm and t2 being a period in seconds after thetriggering of the alarm.

[0032] At step 206, the video image data recorded for the period from t1to t2 is digitized by the A/D converter 128 and supplied to a processor130, which may be or include a computer vision system 132. At step 208,the digitized video image data is analyzed by the processor 130 and/orcomputer vision system 132 according to a set of predetermined criteria,such as any one or more of the criteria discussed above. At step 210,each video frame in the digitized video image data is ranked accordingto how well it meets the predetermined criteria. At step 212, apredetermined number n of video frames having the best ranking arepreferably compressed by the compression means 124 prior to beingtransmitted by the transmitting means 102 a to the remote monitoringstation 112. As discussed above, n is preferably 1.

[0033] Those skilled in the art will appreciate that the system andmethods of the present invention provide the ability to filter out falsealarms by transmitting the most useful image from a set of CCTV camerasto a remote monitoring station where the monitoring staff (or localpolice) can inspect it and make a decision on whether or not the alarmis false.

[0034] While there has been shown and described what is considered to bepreferred embodiments of the invention, it will, of course, beunderstood that various modifications and changes in form or detailcould readily be made without departing from the spirit of theinvention. It is therefore intended that the invention be not limited tothe exact forms described and illustrated, but should be constructed tocover all modifications that may fall within the scope of the appendedclaims.

What is claimed is:
 1. A security monitoring system comprising: an alarmsystem having means for detection of an alarm in a structure; at leastone camera for capturing image data inside and/or outside the structure;a processor for selecting a subset of the image data upon the occurrenceof the alarm based on a set of predetermined criteria; and transmissionmeans for transmitting the subset of image data to a remote location. 2.The security monitoring system of claim 1, wherein the at least onecamera is a video camera, the image data is video image data, and thesubset of the image data is at least one video frame of the video imagedata.
 3. The security monitoring system of claim 1, wherein theprocessor is a computer vision system for analyzing the image dataaccording to the predetermined criteria.
 4. The security monitoringsystem of claim 3, further comprising an image recording system forrecording the image data to be analyzed by the computer vision system.5. The security monitoring system of claim 4, wherein the image data isrecorded for a predetermined time period before and after the occurrenceof the alarm.
 6. The security monitoring system of claim 4, wherein theimage data is analog image data, the system further comprising an analogto digital converter for converting the analog image data to digitalimage data prior to being analyzed by the computer vision system.
 7. Thesecurity monitoring system of claim 1, further comprising a compressionmeans for compressing the subset of the image data prior to transmissionto the remote location.
 8. The security monitoring system of claim 1,wherein the transmission means is a modem for transmitting the subset ofimage data to a remote location over a data link operatively connectedto the modem and the remote location.
 9. The security monitoring systemof claim 1, wherein the at least one camera is a video camera, the imagedata is video image data, and the subset of the image data is at leastone video frame of the video image data, wherein the processor rankseach video frame from the image data according to how well each videoframe meets the predetermined criteria and the transmission meanstransmits a predetermined number of video frames having the best rank tothe remote location.
 10. The security monitoring system of claim 1,wherein the at least one camera is a video camera, the image data isvideo image data, and the subset of the image data is at least one videoframe of the video image data, wherein the processor subtracts anestablished background from each video frame resulting in a differenceregion, and wherein the predetermined criteria are selected from a groupconsisting of: how centered the difference region is in the video frame;how large the difference region is in the video frame; whether thedifference region consists of a large difference region or a group ofsmaller difference regions in the video frame; the contrast of thedifference region in the video frame; the lighting condition on thedifference region in the video frame; and whether a face is detected inthe difference region in the video frame.
 11. The security monitoringsystem of claim 10, wherein if the set of predetermined criteriaincludes whether a face is detected in the difference region of thevideo frame, a further set of predetermined criteria is selected from agroup consisting of: how much of the face is visible in the video frame;whether the face is turned towards or away from the video frame; andwhether key features of the face are visible in the video frame.
 12. Thesecurity monitoring system of claim 1, wherein the at least one camerais a video camera, the image data is video image data, and the subset ofthe image data is at least one video frame of the video image data, andwherein the predetermined criteria are selected from a group consistingof: whether the video frame is blurred; how much skin color is containedin the video frame; if a person is recognized in the video frame; andthe lighting condition on a region of motion in the video frame.
 13. Amethod for monitoring a structure, the method comprising the steps of:detecting an alarm in the structure; capturing image data inside and/oroutside the structure; selecting a subset of the image data upon theoccurrence of the alarm based on a set of predetermined criteria; andtransmitting the subset of image data to a remote location.
 14. Themethod of claim 13, further comprising the step of recording the imagedata.
 15. The method of claim 14, wherein the recording step comprisesrecording the image data for a predetermined time period before andafter the occurrence of the alarm.
 16. The method of claim 14, whereinthe image data is analog image data, the method further comprising thestep of converting the analog image data to digital image data.
 17. Themethod of claim 13, further comprising the step of compressing thesubset of the image data prior to transmission to the remote location.18. The method of claim 13, wherein the image data is video image dataand the subset of the image data is at least one video frame of thevideo image data, wherein the method further comprises the step ofranking each video frame from the image data according to how well eachvideo frame meets the predetermined criteria and the transmission meanstransmits a predetermined number of video frames having the best rank tothe remote location.
 19. The method of claim 13, wherein the image datais video image data and the subset of the image data is at least onevideo frame of the video image data, wherein the method furthercomprises the step of subtracting an established background from eachvideo frame resulting in a difference region, and wherein thepredetermined criteria are selected from a group consisting of: howcentered the difference region is in the video frame; how large thedifference region is in the video frame; whether the difference regionconsists of a large difference region or a group of smaller differenceregions in the video frame; the contrast of the difference region in thevideo frame; the lighting condition on the difference region in thevideo frame; and whether a face is detected in the difference region inthe video frame.
 20. The method of claim 19, wherein if thepredetermined criteria includes whether a face is detected in thedifference region of the video frame, a further set of predeterminedcriteria are selected from a group consisting of: how much of the faceis visible in the video frame; whether the face is turned towards ofaway from the video frame; and whether key features of the face arevisible in the video frame.
 21. The method of claim 1, wherein the imagedata is video image data and the subset of the image data is at leastone video frame of the video image data, and wherein the set ofpredetermined criteria are selected from a group consisting of: whetherthe video frame is blurred; how much skin color is contained in thevideo frame; if a person is recognized in the video frame; and thelighting condition on a region of motion in the video frame.